This invention relates to the computer-based generation of three-dimensional (3D) models of objects defined by two-dimensional (2D) image data.
A traditional image, such as a photograph, displays a 3D object in two dimensions. For example, a digitized photograph represents an object with two-dimensional arrays of pixel values. Efforts have been made to construct a 3D geometric model of an object from 3D image data that has been derived from a traditional 2D representation of the object. Computer-derived 3D image data can be used to render images of the object from a variety of viewpoints and find application in a wide variety of fields including computer animation, virtual reality applications, computer games, movies, environmental modeling, and medicine.
The computer system illustrated in FIG. 2 represents a typical hardware setup for running software that allows a user to generate 3D models from 2D images. The computer system of FIG. 2 includes a CPU 20, RAM 21, and an input/output (I/O) controller 23 coupled by a CPU bus 22. The I/O controller 23 also is coupled by an I/O bus 24 to I/O devices such as a keyboard 25, a mouse 26, a camera 27, and a monitor 28.
In one aspect, generally, a three-dimensional model of an object can be generated using a computer system by identifying reference points in two or more 2D representations of the object to be modelled, and deriving 3D positional data based on an application of an initial estimation model to the identified reference points. A texture-mapped version of the 3D model can be generation by mapping texture information from the 2D representations onto the derived 3D positional data.
In another aspect, generally, a computer-implemented method for generating 3D image data descriptive of an object includes receiving 2D image data for at least two input images of an object, each input image representing a different viewpoint of the object. 2D position data for reference points of the object are determined from the 2D image data, each of the reference points being included in at least two of the input images. If camera parameters for the input images are not known, (i) camera parameters for each of the input images are derived, and (ii) 3D position data for the reference points are derived, based on the 2D position data for the reference points of the input images. A 3D texture-mapped model of the object is created by texture mapping from the 2D image data for the input images using the camera parameters for the input images and the 3D position data for reference points of the object.
In another aspect, generally, a computer-implemented method for generating 3D image data of a head includes receiving 2D image data for at least two input photographs of a head, each input photograph corresponding to a different viewpoint of the head. 2D position data for reference points of the head are determined from the 2D image data based on received user input indicating reference points in the input photographs, each of the reference points being included in at least two of the input photographs. Based on the 2D position data for the reference points of the input photographs, camera parameters for each of the input photographs and 3D position data for the reference points are derived using a non-linear optimization method beginning with an initial approximation. The non-linear optimization method includes determining a current approximation for camera parameters for input images and 3D positions for reference points based on the initial approximation. The following procedures are repeated iteratively until a termination criterion is met: (i) determining an error metric based on reconstructed 2D position data for reference points of input images based on the current approximation, and (ii) updating the current approximation. Then, 3D image data for the head are determined by texture mapping from the 2D image data for the input photographs, using the camera parameters for the input photographs and the 3D position data for reference points of the heads; and a 2D image is rendered from the 3D image data Advantages of the invention include the ability to generate high-quality, low-polygon count 3D image data for an object represented in two dimensions. Moreover, conventional hardware may be used to generate the 3D image data for the objects.
Other advantages and features will become apparent from the following description and from the claims.